Skip to content

feat(customs): add maafw custom - #13

Merged
kqcoxn merged 1 commit into
MaaXYZ:mainfrom
huzesama:feat/onnx-detect-IoU
Aug 20, 2026
Merged

feat(customs): add maafw custom#13
kqcoxn merged 1 commit into
MaaXYZ:mainfrom
huzesama:feat/onnx-detect-IoU

Conversation

@huzesama

@huzesama huzesama commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

根据MaaFramework内置NeuralNetworkDetect修改的神经网络检测(IoU)。解决检测存在全屏大目标时,细分的小目标少甚至没有的问题。适用于需要搭配用户自定义优先级排序进行细分检测的场景。

Summary by Sourcery

添加一个 MaaFramework 自定义 ONNX 识别模块,使用基于 IoU 的抑制策略,以保留重叠的小目标检测,并支持按优先级驱动的结果选择。

New Features:

  • 添加一个基于 ONNX 的自定义识别,用于面向 IoU 的神经网络检测,改善小目标与大目标重叠场景下的检测效果。
  • 支持可配置的置信度与 IoU 阈值、期望标签或类别索引、结果排序,以及按索引选择结果。
  • 提供示例流水线和使用文档,用于集成该自定义识别模块。

Enhancements:

  • 提供 ONNX 模型标签元数据解析和检测结果排序,实现与内置识别行为兼容。

Documentation:

  • 记录该自定义识别模块的配置选项、依赖项、文件布局以及示例用法。
Original summary in English

Summary by Sourcery

Add a MaaFramework custom ONNX recognition that uses IoU-based suppression to preserve overlapping small-object detections and supports priority-driven result selection.

New Features:

  • Add an ONNX-based custom recognition for IoU-oriented neural-network detection, improving detection of smaller objects overlapping larger ones.
  • Support configurable confidence and IoU thresholds, expected labels or class indices, result ordering, and indexed result selection.
  • Include a sample pipeline and usage documentation for integrating the custom recognition.

Enhancements:

  • Provide ONNX model label metadata parsing and detection result sorting compatible with built-in recognition behavior.

Documentation:

  • Document configuration options, dependencies, file layout, and example usage for the custom recognition.

根据MaaFramework内置NeuralNetworkDetect修改的神经网络检测(IoU)。解决检测存在全屏大目标时,细分的小目标少甚至没有的问题。适用于需要搭配用户自定义优先级排序进行细分检测的场景。

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - 我发现了 1 个问题,并留下了一些整体反馈:

  • analyze 中,当 raw_expected 被设置但 expected_indices 为空时处理该情况的代码块,括号/缩进似乎不匹配,按当前形式可能无法编译;请重新格式化该处返回的 AnalyzeResult,以确保语法正确。
  • analyze 中,本地变量 labels 从未被使用,而从 custom_recognition_param 获取的值被直接传入 _load;建议要么统一使用解析得到的 labels,要么移除这个未使用的变量,以避免混淆。
  • main() 中的 CLI 使用说明提到的是 my_agent.py,但实际文件名是 onnxDetect.py;建议更新使用说明文本以与真实文件名一致,从而让入口更加清晰。
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
-`analyze` 中,当 `raw_expected` 被设置但 `expected_indices` 为空时处理该情况的代码块,括号/缩进似乎不匹配,按当前形式可能无法编译;请重新格式化该处返回的 `AnalyzeResult`,以确保语法正确。
-`analyze` 中,本地变量 `labels` 从未被使用,而从 `custom_recognition_param` 获取的值被直接传入 `_load`;建议要么统一使用解析得到的 `labels`,要么移除这个未使用的变量,以避免混淆。
- `main()` 中的 CLI 使用说明提到的是 `my_agent.py`,但实际文件名是 `onnxDetect.py`;建议更新使用说明文本以与真实文件名一致,从而让入口更加清晰。

## Individual Comments

### Comment 1
<location path="Storage/customs/huzesama/onnx-detect-IoU/onnxDetect.py" line_range="132" />
<code_context>
+
+		candidates = self._parse_outputs(outputs, scale, pad_left, pad_top, conf_threshold, roi)
+
+		#expected label 字符串或下标 int 混合 ----
+		expected_indices = self._resolve_expected(raw_expected, self._labels)
+
</code_context>
<issue_to_address>
**suggestion (bug_risk):**`expected_indices` 使用 set 会丢失用户指定的顺序,这会导致 `order_by='Expected'` 相对于输入而言变得不确定。

由于 `_resolve_expected` 返回的是一个 `set`,在 `order_boxes(..., ...)` 调用中对其执行 `list(expected_indices)` 会产生任意顺序,从而破坏与用户 `raw_expected` 的对齐。如果 `order_by='Expected'` 旨在遵循原始顺序,那么 `_resolve_expected` 应当返回一个有序的集合(例如 list 或具有插入顺序的结构),或者应当直接基于 `raw_expected` 派生 `order_map`,同时过滤出合法下标。

建议的实现方式:

```python
		candidates = self._parse_outputs(outputs, scale, pad_left, pad_top, conf_threshold, roi)

		# expected label 字符串或下标 int 混合 ----
		expected_indices = self._resolve_expected(raw_expected, self._labels)

```

```python
def order_boxes(boxes: list[dict], order_by: str, expected_indices: list[int] | None = None) -> list[dict]:
	if order_by == "Horizontal":
		# x 升序,同 x 按 y 升序
		return sorted(boxes, key=lambda b: (b["x"], b["y"]))
	elif order_by == "Vertical":
		# y 升序,同 y 按 x 升序
		return sorted(boxes, key=lambda b: (b["y"], b["x"]))
	elif order_by == "Score":

```

```python
def _resolve_expected(raw_expected, labels):
    """
    Resolve user-provided expected labels/indices into an ordered list of indices.

    Preserves the order in `raw_expected`, filters to valid indices, and de-duplicates
    while keeping the first occurrence of each index.
    """
    resolved: list[int] = []
    seen: set[int] = set()

    for e in raw_expected:
        idx: int | None = None

        # 支持字符串标签或整型下标混合输入
        if isinstance(e, int):
            # 只接受合法范围内的下标
            if 0 <= e < len(labels):
                idx = e
        elif isinstance(e, str):
            # 根据标签名称查找下标
            try:
                idx = labels.index(e)
            except ValueError:
                idx = None
        else:
            # 不支持的类型直接跳过
            idx = None

        if idx is not None and idx not in seen:
            seen.add(idx)
            resolved.append(idx)

    return resolved

```

```python
ordered_candidates = order_boxes(candidates, order_by, expected_indices)

```

我目前只能看到文件的一部分,所以你需要:

1. 用你实际使用的映射逻辑替换 `_resolve_expected` 中的占位实现(`...`),但保留有序的 `resolved` 列表以及 `seen` 集合的模式,以便在保留顺序的同时确定性地处理重复项。
2. 确保所有期望 `_resolve_expected` 返回 `set` 的调用点都更新为与返回 list 的行为匹配(例如移除任何 `list(expected_indices)` 包装,并调整所有依赖 set 特性的操作)。
3. 如果 `raw_expected` 可能为 `None` 或空,你可能希望在调用 `_resolve_expected` 之前处理这种情况(例如在 `raw_expected` 为假值时设置 `expected_indices = []`),以确保 `order_boxes` 一直接收的是一个 list。
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
帮我变得更有用!请在每条评论上点 👍 或 👎,我会根据这些反馈改进以后的代码审查。
Original comment in English

Hey - I've found 1 issue, and left some high level feedback:

  • In analyze, the block handling the case where raw_expected is set but expected_indices is empty appears to have mismatched parentheses/indentation and may not compile as-is; please reformat that AnalyzeResult return to ensure syntactic correctness.
  • The labels local variable in analyze is never used and the value from custom_recognition_param is passed directly into _load; consider either using the parsed labels consistently or removing the unused variable to avoid confusion.
  • The CLI usage message in main() refers to my_agent.py, but the file is named onnxDetect.py; updating the usage text to match the actual filename will make the entry point clearer.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `analyze`, the block handling the case where `raw_expected` is set but `expected_indices` is empty appears to have mismatched parentheses/indentation and may not compile as-is; please reformat that `AnalyzeResult` return to ensure syntactic correctness.
- The `labels` local variable in `analyze` is never used and the value from `custom_recognition_param` is passed directly into `_load`; consider either using the parsed `labels` consistently or removing the unused variable to avoid confusion.
- The CLI usage message in `main()` refers to `my_agent.py`, but the file is named `onnxDetect.py`; updating the usage text to match the actual filename will make the entry point clearer.

## Individual Comments

### Comment 1
<location path="Storage/customs/huzesama/onnx-detect-IoU/onnxDetect.py" line_range="132" />
<code_context>
+
+		candidates = self._parse_outputs(outputs, scale, pad_left, pad_top, conf_threshold, roi)
+
+		#expected label 字符串或下标 int 混合 ----
+		expected_indices = self._resolve_expected(raw_expected, self._labels)
+
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Using a set for `expected_indices` loses the user-specified order, which makes `order_by='Expected'` non-deterministic with respect to the input.

Since `_resolve_expected` returns a `set`, `list(expected_indices)` in the `order_boxes(..., ...)` call will produce an arbitrary order, breaking alignment with the user’s `raw_expected`. If `order_by='Expected'` is meant to respect the original order, `_resolve_expected` should return an ordered collection (e.g., list or insertion-ordered structure), or `order_map` should be derived directly from `raw_expected` while filtering to valid indices.

Suggested implementation:

```python
		candidates = self._parse_outputs(outputs, scale, pad_left, pad_top, conf_threshold, roi)

		# expected label 字符串或下标 int 混合 ----
		expected_indices = self._resolve_expected(raw_expected, self._labels)

```

```python
def order_boxes(boxes: list[dict], order_by: str, expected_indices: list[int] | None = None) -> list[dict]:
	if order_by == "Horizontal":
		# x 升序,同 x 按 y 升序
		return sorted(boxes, key=lambda b: (b["x"], b["y"]))
	elif order_by == "Vertical":
		# y 升序,同 y 按 x 升序
		return sorted(boxes, key=lambda b: (b["y"], b["x"]))
	elif order_by == "Score":

```

```python
def _resolve_expected(raw_expected, labels):
    """
    Resolve user-provided expected labels/indices into an ordered list of indices.

    Preserves the order in `raw_expected`, filters to valid indices, and de-duplicates
    while keeping the first occurrence of each index.
    """
    resolved: list[int] = []
    seen: set[int] = set()

    for e in raw_expected:
        idx: int | None = None

        # 支持字符串标签或整型下标混合输入
        if isinstance(e, int):
            # 只接受合法范围内的下标
            if 0 <= e < len(labels):
                idx = e
        elif isinstance(e, str):
            # 根据标签名称查找下标
            try:
                idx = labels.index(e)
            except ValueError:
                idx = None
        else:
            # 不支持的类型直接跳过
            idx = None

        if idx is not None and idx not in seen:
            seen.add(idx)
            resolved.append(idx)

    return resolved

```

```python
ordered_candidates = order_boxes(candidates, order_by, expected_indices)

```

I only see part of the file, so you’ll need to:

1. Replace the placeholder body in `_resolve_expected` (`...`) with the actual mapping logic you’re using, but keep the ordered `resolved` list and `seen` set pattern so order is preserved and duplicates are handled deterministically.
2. Ensure all call sites that expect `_resolve_expected` to return a `set` are updated to work with a list (e.g., remove any `list(expected_indices)` wrapping and adjust any set-specific operations).
3. If `raw_expected` can be `None` or empty, you may want to handle that before calling `_resolve_expected` (e.g., `expected_indices = []` when `raw_expected` is falsy) so `order_boxes` receives a list consistently.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.


candidates = self._parse_outputs(outputs, scale, pad_left, pad_top, conf_threshold, roi)

#expected label 字符串或下标 int 混合 ----

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk):expected_indices 使用 set 会丢失用户指定的顺序,这会导致 order_by='Expected' 相对于输入而言变得不确定。

由于 _resolve_expected 返回的是一个 set,在 order_boxes(..., ...) 调用中对其执行 list(expected_indices) 会产生任意顺序,从而破坏与用户 raw_expected 的对齐。如果 order_by='Expected' 旨在遵循原始顺序,那么 _resolve_expected 应当返回一个有序的集合(例如 list 或具有插入顺序的结构),或者应当直接基于 raw_expected 派生 order_map,同时过滤出合法下标。

建议的实现方式:

		candidates = self._parse_outputs(outputs, scale, pad_left, pad_top, conf_threshold, roi)

		# expected label 字符串或下标 int 混合 ----
		expected_indices = self._resolve_expected(raw_expected, self._labels)
def order_boxes(boxes: list[dict], order_by: str, expected_indices: list[int] | None = None) -> list[dict]:
	if order_by == "Horizontal":
		# x 升序,同 x 按 y 升序
		return sorted(boxes, key=lambda b: (b["x"], b["y"]))
	elif order_by == "Vertical":
		# y 升序,同 y 按 x 升序
		return sorted(boxes, key=lambda b: (b["y"], b["x"]))
	elif order_by == "Score":
def _resolve_expected(raw_expected, labels):
    """
    Resolve user-provided expected labels/indices into an ordered list of indices.

    Preserves the order in `raw_expected`, filters to valid indices, and de-duplicates
    while keeping the first occurrence of each index.
    """
    resolved: list[int] = []
    seen: set[int] = set()

    for e in raw_expected:
        idx: int | None = None

        # 支持字符串标签或整型下标混合输入
        if isinstance(e, int):
            # 只接受合法范围内的下标
            if 0 <= e < len(labels):
                idx = e
        elif isinstance(e, str):
            # 根据标签名称查找下标
            try:
                idx = labels.index(e)
            except ValueError:
                idx = None
        else:
            # 不支持的类型直接跳过
            idx = None

        if idx is not None and idx not in seen:
            seen.add(idx)
            resolved.append(idx)

    return resolved
ordered_candidates = order_boxes(candidates, order_by, expected_indices)

我目前只能看到文件的一部分,所以你需要:

  1. 用你实际使用的映射逻辑替换 _resolve_expected 中的占位实现(...),但保留有序的 resolved 列表以及 seen 集合的模式,以便在保留顺序的同时确定性地处理重复项。
  2. 确保所有期望 _resolve_expected 返回 set 的调用点都更新为与返回 list 的行为匹配(例如移除任何 list(expected_indices) 包装,并调整所有依赖 set 特性的操作)。
  3. 如果 raw_expected 可能为 None 或空,你可能希望在调用 _resolve_expected 之前处理这种情况(例如在 raw_expected 为假值时设置 expected_indices = []),以确保 order_boxes 一直接收的是一个 list。
Original comment in English

suggestion (bug_risk): Using a set for expected_indices loses the user-specified order, which makes order_by='Expected' non-deterministic with respect to the input.

Since _resolve_expected returns a set, list(expected_indices) in the order_boxes(..., ...) call will produce an arbitrary order, breaking alignment with the user’s raw_expected. If order_by='Expected' is meant to respect the original order, _resolve_expected should return an ordered collection (e.g., list or insertion-ordered structure), or order_map should be derived directly from raw_expected while filtering to valid indices.

Suggested implementation:

		candidates = self._parse_outputs(outputs, scale, pad_left, pad_top, conf_threshold, roi)

		# expected label 字符串或下标 int 混合 ----
		expected_indices = self._resolve_expected(raw_expected, self._labels)
def order_boxes(boxes: list[dict], order_by: str, expected_indices: list[int] | None = None) -> list[dict]:
	if order_by == "Horizontal":
		# x 升序,同 x 按 y 升序
		return sorted(boxes, key=lambda b: (b["x"], b["y"]))
	elif order_by == "Vertical":
		# y 升序,同 y 按 x 升序
		return sorted(boxes, key=lambda b: (b["y"], b["x"]))
	elif order_by == "Score":
def _resolve_expected(raw_expected, labels):
    """
    Resolve user-provided expected labels/indices into an ordered list of indices.

    Preserves the order in `raw_expected`, filters to valid indices, and de-duplicates
    while keeping the first occurrence of each index.
    """
    resolved: list[int] = []
    seen: set[int] = set()

    for e in raw_expected:
        idx: int | None = None

        # 支持字符串标签或整型下标混合输入
        if isinstance(e, int):
            # 只接受合法范围内的下标
            if 0 <= e < len(labels):
                idx = e
        elif isinstance(e, str):
            # 根据标签名称查找下标
            try:
                idx = labels.index(e)
            except ValueError:
                idx = None
        else:
            # 不支持的类型直接跳过
            idx = None

        if idx is not None and idx not in seen:
            seen.add(idx)
            resolved.append(idx)

    return resolved
ordered_candidates = order_boxes(candidates, order_by, expected_indices)

I only see part of the file, so you’ll need to:

  1. Replace the placeholder body in _resolve_expected (...) with the actual mapping logic you’re using, but keep the ordered resolved list and seen set pattern so order is preserved and duplicates are handled deterministically.
  2. Ensure all call sites that expect _resolve_expected to return a set are updated to work with a list (e.g., remove any list(expected_indices) wrapping and adjust any set-specific operations).
  3. If raw_expected can be None or empty, you may want to handle that before calling _resolve_expected (e.g., expected_indices = [] when raw_expected is falsy) so order_boxes receives a list consistently.

@kqcoxn
kqcoxn merged commit fe8cb22 into MaaXYZ:main Aug 20, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants